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Abstract .  The  concepts  and  results  of  [l]  are  discussed  in  the  light  of 
a  new  result  on  the  optimality  of  the  minirnTjan  residual  algorithm.  The 
proof  is  given  in  Section  3.  The  terminology  of  [l]  is  described  in 
Section 


i 
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1.  Svunmary 

The  conjugate  gradient  algorithm  (called  CG  hereafter)  is  a  pop\ilar 
way  to  solve  large  sparse  positive  definite  systems  of  equations.  The 
minimum  residual  algorithm  (called  MR)  is  closely  related  to  CG  and  can 
he  applied  to  any  nonsingular  system,  ,  [ 6 ] .  In  this  summary  we  describe 'i 

our  result  and,  of  more  importance,  comment  on  their  significance.  The 
discussion  is  confined  to  exact  arithmetic  because  this  enquiry  concerns 
only  the  theory  of  MR  ,  CG  and  related  algorithms ,  not  their 
implementatioi^ 

Both  CG  and  MR  require  between  1  and  n  steps  to  solve  a  given 

n  by  n  nonsingular  system  Ax  =  b  ,  and  both  extremes  can  occiir.  One 

of  the  attractions  of  these  methods  is  that  they  are  finite  and  only  use 

A  to  form  products  Av  ,  Moreover  it  is  known  that  both  CG  and  MR 

are  optimal  in  a  certain  sense.  More  precisely,  at  each  step  each  method 

produces  a  vector  z  which  minimizes  the  norm  of  the  residual  vector 

b  -  Az  over  all  z  in  an  appropriate  subspace  of  .  As  usual 

denotes  the  space  of  all  real  n-dimensional  column  vectors.  The  methods 

differ  only  in  their  choice  of  norm,  MR  uses  the  Euclidean  norm  llvll 

—1/2 

whereas  CG  uses  an  "energy"  norm  lA  vB  .  The  former  is  independent 
of  A  but  the  latter  has  meaning  only  for  symmetric  positive  definite 
matrices  A  .  For  the  sake  of  simplicity  and  generality  we  shal.1  concentrate 
on  MR  in  the  rest  of  this  essay. 

Most  numerical  analysts  suppose  that  the  general  theory  of  MR  stops 
at  this  point.  Indeed,  to  estimate  how  many  steps  will  be  required  to 
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achieve  a  prescribed  reduction  in  the  residual  norm  seme  additional 
information  about  A  ,  such  as  its  condition  nimber  or  seme  other 
measure  of  its  eigenvalue  distribution,  must  be  supplied. 

Recently,  in  [l],  Traub  and  Wozniahowski  have  sought  to  enlarge  the 
scope  of  the  discussion  of  MR  .  Following  the  lines  laid  down  in  [2] 
they  remove  the  restriction  of  algorithms  to  the  standard  (polynomial)  class 
over  which  MR  and  CG  are  optimal.  Why,  they  say,  restrict  attention 
to  the  standard  (Krylov^)  subspace  of  ? 

Will  MR  remain  optimal  when  any  rival  algorithm  is  allowed?  They 
address  this  question  in  [l]  while  erecting  a  new  framework  around  the 
study, in  general,  of  iterative  algorithms  for  solving  Ax  =  b  .  They  succeed 
in  showing  that  MR  costs  at  most  one  iteration-step  more  than  an  optimum 
algorithm  could  require. 

Actually  MR  is  still  optimal  for  positive  definite  A  ;  this  note 
will  explain  why,  and  then  go  on  to  indicate  how  all  such  results,  theirs 
and  ours,  can  mislead  the  unwary  reader. 


2,  Discussion  of  Ref,  [l] 

To  go  any  further  we  must  introduce  some  terminology.  At  step  J 

MR  computes  a  linear  combination  x,  of  the  vectors 

J 

b  ,  Ab  ,  A^b,  ...,  A’^”^  .  No  generality  is  lost  by  the  assumption  (in 
force  throrighout  the  paper)  that  this  set  of  vectors  is  linearly  independent 


Krylov  subspaces  Eure  defined  in  Section  2. 
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Another  vector  A'^b  ,  or  something  equivalent  to  it ,  is  needed  to  determine 


the  coefficients  for  MR's  output  x  although  A'^b  itself  is  not  part 


of  the  linear  combination.  For  future  reference  we  let  v  denote  x  's 

J  J 


residual  norm.  The  defining  property  of  MR  and  an  expression  for  x 


i 


are  given  in  the  following  equations , 


Vj  Sminllb  -  YQAb  -  Yj^A^b  -  ...  -  Yj_^A'^b||  , 

=  minllb  -  ACY^jb  +  Y^^Ab  +  ...  +  Yj_^A’^"^)ll  , 

"'^i 

=  lb  -  Ax_^l  . 

The  theory  in  [l]  can  be  motivated  by  asking  whether  there  exists  some 
clever,  well-hidden  algorithm  which,  with  no  further  multiplication  of 
vectors  by  A  ,  delivers  a  vector  z  with  lb  -  Azl  <  and  so  surpasses 

MR  .  The  answer,  after  a  little  thought,  is  yes.  The  "well  hidden" 
algorithm  is  Gaussian  elimination  (with  pivoting)  which  delivers  A  ^ 
itself  without  computing  Av  for  any  v  I  Of  course  we  feel  cheated  and 

this  reaction  shows  that  the  question  must  be  posed  with  more  care. 

Rival  algorithms  must  be  prevented  fron  getting  at  A  through  the  back 
door. 

One  way  to  be  fair  is  to  limit  knowledge  of  A  to  the  current 
information:  b  ,  Ab,  ...,  A'^b  ,  or  some  "equivalent"  set  of  vectors. 

In  general  these  vectors  do  not  fix  A  uniquely  and  so  there  is  the  set 


=  (A:  A^b  =  A^b 


i  =  1, 


■  ,  J} 


of  matrices  indistinguishable  from  A  with  the  given  information.  Sometimes 
we  restrict  to  a  particular  class  of  matrices  A  ,  sometimes  not. 

The  context  decides. 

Trauh  and  Wozniakowski  use  A*^  in  specifying  the  cost  of  an  algorithm 
for  a  given  matrix.  The  technical  description  is  given  in  Section  U  (ad.ong 
with  a  warning)  but,  in  English,  the  cost  is  the  minimal  nianber  of  steps  j 
required  to  achieve  a  given  reduction  in  the  residual  norm  for' any  A  in 
A*^  ,  i.e.  the  number  of  steps  needed  in  the  worst  case.  This  cost,  for 
an  algorithm  (jj  ,  is  denoted  by  k((j).  A)  .  The  new  version  of  optimality 
on  which  [l]  is  based  is  optimality  over  a  whole  class  F  of  matrices. 

The  authors  seek  an  algorithm  $  ,  which  satisfies 

k($.  A)  =  min  k((j).  A)  for  each  A  in  F  . 

The  minim'um  is  over  all  algorithms  using  the  given  information. 

These  aire  the  basic  concepts.  There  are  many  theorems  in  [l].  The  main 
res-ult,  as  mentioned  above,  is  that  over  the  class  F  =  SPD  ,  the  class 
of  symmetric  positive  definite  matrices ,  and  also  over  larger  classes  F  , 

MR  is  within  one  matrix-vector  multiplication  of  being  an  optimal  algorithm; 
but  [l]  does  not  exhibit  an  cilgorithm  $  that  is  optimal  for  any  of 
those  classes  F  .  Instead  they  study  restricted  classes  and  finish 
[l]  with  conJect\ires  and  open  problems,  as  if  the  whole  field  were  ripe 
for  further  development. 

We  doubt  the  ripeness.  The  questions  being  asked  seem  to  be  the 
wrong  questions.  For  a  start  the  quest  for  many  optimal  $  is  not  necessary 


because 
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MR  i£  optimal  for  the  class  SPD  , 
and  for  any  larger  class. 


The  proof  is  given  in  Section  3.  It  amounts  to  shoving  that  for  each 


rival  vector  y  x.  one  can  construct  a  positive  definite  5  =  A(y) 


in 


aJ 


such  that 


llh  -  Ay  11  > 


Thus  y  is  worse  than  Xj  for  A  and  for  many  other  matrices  in  . 


Even  worse,  for  y  not  in  the  usual  suhspaces,  {lib  -  ^11  :  A  E.  A.^  H  SPD} 
is  unbounded. 

The  devastating  consequence  of  this  observation  for  the  theory 
launched  by  Traub  and  Wozniakowski  is  that  if  ())  produces  ajay  approximation 
unrestricted  to  combinations  of  b  ,  Ab,  A*^”^  then  for  A's  in 

SPD  the  cost  k(<l).  A)  >  n  . 

In  other  words  MR  wins  because  all  its  new  rivals  die  at  the  starting  gate 
The  villain  of  the  piece  is  the  class  SPD  .  It  is  so  big  that  the  generality, 
espoused  in  [l],  of  allowing  any  €dgorithm  4i  (euid  hence  approximations 
outside  the  usual  subspace)  is  annulled  by  the  required  fairness  of  taking 
the  worst  case  in  A*^  .  The  theory  becomes  vacuous. 

On  the  other  hand  SPD  is  the  most  important  class  of  matrices  in  the 
application  of  iterative  methods. 

One  way  to  try  to  save  the  theory  is  to  restrict  it  to  small  subsets 
of  SPD  ,  for  example  to  those  matrices  with  condition  number  <  100  .  Yet 
even  here  k((^.  A)  =  n  unless  (p  is  very  close  to  MR  ,  as  shown  in  Section  3. 
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In  those  cases  when  k((j).  A)  <  n  the  theory  in  [l],  though  not  vacuotis, 
reduces  to  a  translation  into  its  terminology  of  standard  bomds,  see  [1+] 
for  example,  on  convergence  rates  of  MR  .  How  sad  it  is  to  evade  the 
Charyhdis  of  vacuity  only  to  founder  on  the  Scylla  of  redundancy."^ 

V/hat  has  gone  wrong?  Why  has  such  an  enticing  research  program,  on 
closer  examination,  dissolved  away  like  the  Cheshire  Cat  in  "Alice  Through 

the  Looking  Glass"?*  We  are  not  sure  but  we  offer  the  following  thought. 

2  1 

The  information  {b,  Ab,  Ab,  ...,  A*^b}  seems  to  arise  as  an  artifact 
of  the  MR  and  CG  algorithms,  not  as  part  of  the  problem  of  solving 
Ax  =  b  by  some  iterative  method.  The  optimality  of  MR  over  SPD  may 
be  true  but  it  is  no  more  exciting  than  the  discovery  that  the  dress  suit 
made  for  Mr.  X  fits  him  better  than  it  does  anyone  else. 

For  readers  of  this  note  who  are  not  familiar  with  [l]  we  have  added 
a  final  section  in  which  its  leading  notions  are  stated,  together  with 
our  comments.  This  will  make  it  clear  that  this  paper  and  [l]  are  concerned 
with  exactly  the  same  problem,  namely  the  analytic  complexity  of  methods 
for  solving  Ax  =  b  which  are  restricted  to  the  use  of  Krylov  information 
on  A  .  Whereas  [l]  claims  that  this  is  a  new  topic  deserving  of 
development  we  suggest  that  the  nonvacuous  aspects  are  well  known ,  although 
they  are  usually  expressed  in  more  mundeme  language  than  is  favored  in  [l]. 

Here,  in  one  sentence,  is  our  version  of  the  analytic  complexity  of 
the  problem.  Over  the  standard  matrix  classes  (SPD,  Sym,  nonsingular), 
among  all  algorithms  using  the  given  information  MR  is  (worst  case) 
optimal  and  its  complexity  (number  of  steps  required)  is  n  ,  which  is 
as  bad  as  it  could  be. 

^From  the  Greek  myths.  Boats  in  the  Strait  of  Messina  had  to  steer 
a  very  fine  line  between  two  monsters  Charybdis  (a  whirlpool)  €uid  Scylla 
( a  rock ) . 
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Unless  worst  case  analyses  yield  something  better  than  worst  possible 
outcomes  our  interest  shifts  from  them  to  the  reasons  why  particiilar  cases 
fare  so  well.  Except  for  special  right  hand  sides  b  the  residual 
reduction  produced  by  MR  depends  entirely  on  the  usually  lanknown 
distribution  of  eigenvalues.  Significant  diminutions  occur  after  a  small 
number  of  steps  when  the  eigenvalues  are  grouped  into  a  small  ntunber  of 
.clusters  —  even  when  the  condition  number  is  large.  It  is  specific 
surprising  results  such  as  this  which  give  substance  to  the  study  of  MR 
and  CG  . 


3.  Optimality  of  MR 

A  little  preparation  is  necessary  before  we  can  state  and  prove 
the  theorem. 

The  only  information  available  concerning  the  nonsingular  n  by  n 

2  1 

matrix  A  is  the  set  of  vectors  {b  ,  Ab  ,  A  b,  ...,  A'^b}  ,  1  <  j  <  n  . 

Actually  the  methods  use  "equivalent"  combinations  of  these  vectors 
which  yield  alternative,  and  more  practical  bases  for  the  subspace 

=  span{b  ,  Ab,  A-^b)  ,  1  <  n  . 

These  subspaces  of  ]R^  are  sometimes  called  Krylov  subspaces . 

In  theory  it  is  possible  to  have  A°H)  dependent  on  b  ,  Ab,  ...,  A™~^ 
for  some  m  <  n  .  In  this  case  it  is  easily  verified  that  K™  is  invariant 
.  Moreover  the  exact  solution  A”^  lies  in  K”  . 


under  A 


Consequently 
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there  is  no  need  to  look  outside  ^  for  an  approximation.  Theoretically 
A  can  be  replaced  by  its  restriction  to  and  the  change  will  not  be 

noticed.  So,  without  loss  of  generality,  we  may  invoke 

Assumption  1.  {b,  Ab,  ...,  A^  is  linearly  independent. 

Now  the  superscripts  on  the  indicate  dimension. 

A  useful  basis  for  {  =  3^^  )  is  the  Lanczos  basis 

{^1  ,  <l2>  •••»  obtained  by  applying  the  Gram-Schmidt  orthonormalizing 

process  to  the  original  ordered  basis  {b  ,  Ab,  ...,  A^~Hi}  .  If  we 
introduce  the  n  by  j  matrix  Qj  =  (q.^  ,  q^,  ...,  qj)  we  have 

T 

QjQj  =  ,  the  j  by  j  identity  matrix,  j  =  1,  ...,  n  .  (2-1) 


3y  representing  A  in  this  basis  our  proof  beccanes  transparent. 

The  actual  details  of  the  MR  algorithm  are  not  important  here.  Its 
output  Xj  ,  at  step  J  ,  depends  on  A'^b  but  x^  itself  lies  in  . 
By  definition  of  MR  Xj  satisfies 


V  —  min  !lb  -  Avll  =  lib  -  Ax, 
J  4  J 

veK-J 


(2-2) 


Note  that  the  information  available  determines  the  action  of  A  on 
but  not  on  since  A*A'^b  is  unknown. 

The  set  of  matrices  indistinguishable  from  A  at  step  J  is 


A*^  =  {A:  Pb  =  A^b 


i  =  1 


9  •  •  •  9 


J}  ,  J  <  n 
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In  other  words,  at  this  step  is  uniquely  determined  but 

is  not  (unless  j  =  n  -  1  )  . 

It  will  be  convenient  to  abbreviate  by  GL  ,  Sym,  and  SPD  the 
classes  of  nonsingtilar,  symmetric,  and  symmetric  positive  definite  matrices. 

The  reason  for  introducing  the  Lanczos  basis  is  that  with  respect  to 
it  any  A  €  Sym  is  represented  by  a  tridiagonal  matrix  .  We  define 


^2  “2 


“j 


(2-3) 


The  preparation  is  over.  We  now  proceed  to  the  statment  and  proof  of 
the  theorem. 

The  optimality  of  MR  for  a  single  matrix  A  has  been  stated  above 
in  (2-2),  but  -MR  is  optimal  in  a  somewhat  broader  sense.  With  repect 
to  n 
of  the 


SPD  there  is  no  vector  in  which  is  better  than  Xj  because 


THEOREM.  min  sup  lib  -  AyU  =  Vj  . 
ySE^  AeA-^nsPB 

We  prove  the  theorem  by  considering  a  special  one-parameter  fsunily 
of  matrices  A^  and  establishing  two  lemmas.  Using  the  orthonormal 
basis  consisting  of  the  columns  of  we  have 


J 
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T 

n  n 


T,  '  0 

J  I 


0  '  U 


(2-U) 


where  •  =  3.  ^  and  is  known,  whereas  U  stands  for  the  unknown  part  of 
T  .  Different  A  ^  A*^  H  SPD  will  have  different  values  for  U  .  We  define 


implicitly  by 


Q-^A  Q  = 
nan 


0  'al 


n-j 


(2-5) 


where  •  =  ^j+1  * 


LEMMA  1.  For  all  large  enough  a  ,  A^  S  O  SPD 


Proof.  Recall  that  is  orthogonal  so  that  A^  S  Sym  ,  by 

construction.  Moreover  A^  €  A*^  by  comparison  of  (2-U)  and  (2-5).  More 

til 

formally,  writing  e^  for  the  i^  column  of  I  ,  we  have 


Vi "  Vj'i  *  J  • 


til 

The  Kronecker  symbol  5^^  is  the  i  element  of  Bj  .  Thus 


A/^  «  AK^  ,  i  <  J  , 


and  so  A^  and  A  produce  the  same  K  for  i  *  1,  ...,  J  +  1 
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It  remains  to  see  when  A  G  SPD  .  Note  that  by  doing  a  block 

a 


triangular  factorization 


T 

5  A  Q 
n  an 


0 

T 

c 

I  , 

0 

n-J 

I 


2  T 

0  '  al  -6^  n.e  e^ 
1  n-j  J+1  j  1  1 


I  ° 


0  '  I 


n-j 


T  T  -1  T  —1 

where  c  =  S-.^e.!.  and  n.  =  e.T  e.  .  In  other  words  A  is 

jjjj  “ 

congruent  to  the  direct  sum 


Tj  ©  diag(a  -  ,  a,  ....  o)  , 


Observe  that 


0  <  n.  <  =  lA  ,„(T,)  <  1A„,„(A) 


'i  i 


min'^j 


min 


So  A^  ©  SPD  when  a  > 


LEMMA  2.  If  y  ^  then  lib  -  A^yll  -►  »  as  a  .  For  each 

y  S  there  is  an  a  such  that  lib  -  A  yll  >v,  for  a  >a  >  “ni 

a  j 

equality  holds  if  an  only  if  y  *  . 


Proof.  Represent  y  in  the  Lanczoe  basis  as 


y 
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where  s  S  ,  t  E  .  Then,  with  b  =  q.^6^  , 


“  1 

•  ' 

II 

k'l»l  -  «n 

• 

s 

•1 

ol  , 

1  n-J 

t 

II 

_  1  ^ 

-  e^a  -  ctt 


,  since  is  orthogonal. 


T  T 

where  t  =  e^t  ,  a  =  e^s  . 
If  y  ^  then  t  ^  0  and 


lib  -  A  yll  ^  allt  +  e^  a/all  > 
a  1 

-♦•  00  ,  as  a 


4  2 

More  precisely  for  any  y  ^  K''  ,  Bb  -  A^yH  is  a  quadratic  in  a  with 

2  2 

leading  coefficient  lltB  and  so  exceeds  Vj  for  all  o  exceeding  some 

ct  depending  on  t  and  s  .  When  t  =  0  then  =  Ay  ,  we  can  take 

5=0,  and  standard  least  squares  theory  states  that  the  minimum  residual 

norm  v  is  attained  when  b  -  Ay  i  .  This  occurs  only  when 
J 

s  =  .  In  that  case 


y  =  xj  .  9j=  ana  .  □ 
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Proof  of  theorem. 

If  y  ^  then  by  Lemmas  1  and  2, 

sup  -  Ayll  >  supllb  -  A^yB  *  ®  . 

A-^nSPD  “ 

If  y  G  then 

sup  Bb  -  ^11  =  Hb  -  Ayll  >  lib  -  Ax  11  =  V  .  □ 

A  J  J 

A‘'nSPD 

We  remark  that  we  have  actually  proved  that  MR  is  optimal  over  any 
class  larger  than  SPD  ,  in  particular  over  Sym  and  GL  . 

Let  us  consider  briefly  what  happens  when  a  is  not  permitted  to  go 
to  “  .  Using  the  expressions  in  the  proof  of  Lemma  2  we  write 
u  =  62.^1  “  find  that 


lib  -  A^yB^  =  llufl^  -  2iiT  +  +  2aaT  +  a^BtB^ 


where  y 

a  <  BaB 

MR  ,  we 
cases  we 


T 

=  eju 


,  T  =  e^t  f 


T 

a  *  e  s 
J 


It  would  be  unreasonable  to  demand 
and  so  we  see  that,  unless  Btll  is  tiny,  i.e.,  (p  is  close  to 
can  have  Bb  -  A^yfl  ^  BAll*BtB  >  the  required  accuracy  .  In  such 
still  have  k(<t).  A)  =  n  and  the  theory  in  [l]  becomes  vacuous. 


U.  The  Complexity  Connection 

We  describe  the  principal  terms  in  [l]. 

The  task  is  to  find  x  in  so  that,  for  given  A  and  b  , 

lb  -  AxB  <  eBbl  ,  for  some  given  fixed  e  .  Thus  e  is  a  parameter  in 
the  theory.  The  information  {b,  Ab,  ...,  A’^b}  is  denoted  by  N.(A,  b)  . 
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The  class  from  which  A  is  drawn  is  F  .  Our  set 

=  {r;  r  e  F  ,  =  A^h  ,  i  =  1,  . . . ,  J } 


is  denoted  hy  V(Nj(A,  b))  .  Recall*  from  [3]  or  [U],  that 

MR  is  optimal  among  those  algorithms  which  use  Nj  and  deliver  output 

in  the  Krylov  subspace  .  The  springboard  for  [l]  is  the  removal  of 

this  restriction  on  rival  algorithms.  We  shall  say  that  an  algorithm 

is  new  if  its  output  using  N  is  not  in  .  Now  we  ajre  ready  to 

0 

consider  the  main  concepts  in  [l]. 

For  any  algorithm  (()  ,  whose  output  fran  Nj  is  defines 


k{<)).  A)  s  iiiin{k;  lib  -  Az,  li  <  cllbl  ,  V  6  A^} 


as  the  matrix  index.  It  purports  to  be  the  cost  of  $  as  measured  by 
the  number  of  matrix- vector  products  needed  in  the  worst  case.  Unfortunately 
an  enlargement  in  F  can  increase  A)  thus  robbing  k((j).  A)  of 

that  meaning.  As  shown  in  Section  2  we  may  assume  that  the  set  Nj  is 
lineatrly  independent  for  all  J  <  n  .  When  j  =  n  the  set  must  be 
dependent  (it  has  n  +  1  vectors)  and  MR  delivers  the  exact  solution. 

Our  theorem  in  Section  2  shows  that  if  F  o  SPD  and  (J)  is  a  new  algorithm 
then  sup  |b  -  Sz.  J  =  »  ,  for  all  k  <  n  .  Hence  k((|).  A)  >n  ! 

^  Ir  * 

.  AEA^ 

For  old  algorithms  \j^  k(4'.  A)  >k(MR,  A)  .  The  value  of  e  is  irrelevant. 
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Next  comes  the  class  index  of  (ji  , 

k((|),  F)  =  sup  k((j).  A)  . 

For  new  algorithms  k(4),  F)  ^n  for  F  —  SPD  and  for  many  smaller  classes 
Even  for  MR,k(MR,  SPD)  =  n  .  Since  k(()).  A)  itself  depends  on  F  the 
distinction  between  k(<{).  A)  and  k(ii>,  F)  is  obscure. 

Traub  and  Wozniakowski  seek  to  minimize  these  indices.  They  define 
the  optimal  matrix  index  by 

k(A)  ®  min  k(<J),  A) 

and  the  optimal  class  index  by 

k(F)  =  max  k(A)  , 

PEF 

By  the  remarks  made  above,  if  F  13  SPD  then 

min  k(<j>.  A)  >  n 
new  0 


while 


min  k(i{).  A)  *  k(MR,  A)  . 
old 


An  algorithm  is  called  strongly  optimal  (over  F  )  if 


k(^.  A)  -  k(A)  ,  VA  e  F 


Psige  l6 


and  is  called  optimal  (over  F  )  if 

k(i,  F)  =  k(F)  . 

We  have  just  seen  that  MR  is  strongly  optimal  over  any  F  D  SPD  . 

It  should  he  mentioned  that  [l]  does  not  exhibit  this  result.  Instead 
a  nice  proof  shows  that  if  F  is  orthogonally  invariant  then 

k(MR,  A)  -  k{A)  <  1  ,  V  A  e  F  . 

In  addition  a  very  contrived  2  by  2  example  is  given  to  show  that 
equality  can  occur  for  a  peculiar*  F  . 

We  want  to  make  a  comment  on  the  hypothesis  that  F  be  orthogonally 
invariant.  One  of  the  attractive  features  of  Krylov  information  N,(A,  b) 
is  that  it  is  coordinate  free.  The  standard  methods  and  theory  are 
geometric  in  nature.  Any  change  of  basis  which  preserves  the  Euclidean 
norm  is  permitted.  It  seems  unnatural  to  introduce  matrix  classes  F 
which  do  not  share  this  property  with  the  information.  It  is  hardly 
stirprising  that  the  study  made  in  [l]  of  tridiagonal  matrices  shows 
that  "anything  can  happen." 

By  studying  subclasses  of  SPD  whose  members  have  condition  numbers 
bounded  by  small  enough  <  it  is  possible  to  obtain  k(<j>.  A)  <  n  ,  VA  €  F 
and  sometimes  k(A)  <  n  .  Of  course  k(MR,  A)  will  then  be  a  complicated 
function  of  e  and  <  .  This  topic  has  been  studied  by  numerical 
analysits.  They  showed  how  cond(A)  can  be  used  to  boiind  k(MR,  A)  . 

With  more  information  on  the  spectrum  more  accurate  bounds  can  be  derived. 


Unfortunately  the  theorems  in  Section  5  on  symmetric  matrices  contain  no 
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new  insights  hut  are  translations  into  the  language  of  complexity  of  the 
known  hounds  mentioned  above.  In  contrast  the  results  on  unsymmetric  matrices 
are  new  and  quite  misleading.  The  authors  prove  in  a  dead  pan  manner  that 
even  when  cQnd(A)  =1  it  is  possible  to  have  k(MR,  A)  =  n  .  Not  a  word 
is  said  about  the  fact  that  for  unsynmetric  matrices  cond(A)  is  no 
indicator  of  the  clustering  of  A's  spectrum  and  so  is  virtually 
irrelevant  to  the  convergence  rate  of  MR  .  Cond(A)  meas\ires  the  spread 
of  singular  values,  not  the  eigenvalues. 

The  theorems  in  [l]  tell  us  more  about  Analytic  Complexity  theory 
than  about  iterative  methods  for  solving  Ax  =  b  . 
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